Use explicit Anthropic computer tools - #16
Conversation
ff4e3e6 to
4383ea5
Compare
|
Firetiger deploy monitoring skipped This PR didn't match the auto-monitor filter configured on your GitHub connection:
Reason: PR modifies AI/computer-use tool modules (@onkernel/cua-ai, @onkernel/cua-agent) rather than kernel API endpoints or Temporal workflows. To monitor this PR anyway, reply with |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Autofix Details
Bugbot Autofix prepared a fix for the issue found in the latest run.
- ✅ Fixed: Exported
createActionSchemafunction lacks TSDoc- Added a TSDoc block to
createActionSchemadescribing Anthropic compatibility and its unsupported-action validation behavior.
- Added a TSDoc block to
Or push these changes by commenting:
@cursor push 07eace1ec7
Preview (07eace1ec7)
diff --git a/packages/ai/src/providers/anthropic/actions.ts b/packages/ai/src/providers/anthropic/actions.ts
--- a/packages/ai/src/providers/anthropic/actions.ts
+++ b/packages/ai/src/providers/anthropic/actions.ts
@@ -37,6 +37,12 @@
return resolved;
}
+/**
+ * Build an Anthropic-compatible action schema for CUA browser actions.
+ *
+ * Pass `actions` to expose only a supported subset. Unsupported actions are
+ * rejected so callers get early validation before tool invocation.
+ */
export function createActionSchema(actions?: readonly CuaActionType[]) {
return createCuaActionSchema(resolveAnthropicActions(actions));
}You can send follow-ups to the cloud agent here.
e517a08 to
b5f3102
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue. You can view the agent here.
Reviewed by Cursor Bugbot for commit b5f3102. Configure here.
06ac4ab to
51a3bdf
Compare
51a3bdf to
c8849b2
Compare
Replace the per-provider switch in resolveCuaRuntimeSpec with a registry lookup over provider modules that conform to a shared CuaProviderModule interface. Adding a provider is now "write the module, add one line"; the `satisfies Record<CuaProvider, CuaProviderModule>` makes an unconformed or missing provider a compile error. - Define CuaProviderModule in providers/common.ts and move the provider contract types (CuaPayloadHook, CuaScreenshotSpec, CuaRuntimeSpec, etc.) there so common.ts owns the contract and runtime-spec.ts only composes it. - Expose coordinateSystem() as a function on each provider instead of the constant-like COMPUTER_TOOL_COORDINATES field. - Standardize on build<Provider>SystemPrompt() everywhere; add the missing buildOpenAISystemPrompt(). - Drop the dead yutoriBuiltinToolsOnPayload alias; yutoriNativeToolSetOnPayload is the single canonical name. Fix yutori/provider.ts to import the payload context type from ../common instead of back-importing runtime-spec. - Preserve Yutori's model-facing toolDefinitions: [] invariant in its module. - Add provider-module.test.ts asserting every provider conforms to the contract; update README to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 participant
Summary
Two related changes to
@onkernel/cua-ai(and its@onkernel/cua-agentconsumer):1. Explicit Anthropic computer tools
computer_batchtool by default for Anthropic, withexcludeBatchfor individual-only tool definitions.CuaAgent/CuaAgentHarnessoptions so providers decide whether batch is part of their runtime defaults.2.
CuaProviderModulecontract + registry refactorswitchinresolveCuaRuntimeSpecwith a registry lookup over provider modules that conform to a sharedCuaProviderModuleinterface. Adding a provider is now "write the module, add one line";satisfies Record<CuaProvider, CuaProviderModule>makes a missing or non-conforming provider a compile error.CuaPayloadHook,CuaScreenshotSpec,CuaRuntimeSpec, etc.) intoproviders/common.tssocommon.tsowns the contract andruntime-spec.tsonly composes it. This also removes the provider →runtime-specback-import (a layering inversion).coordinateSystem()as a function instead of the constant-likeCOMPUTER_TOOL_COORDINATESfield.build<Provider>SystemPrompt()across all providers; add the previously-missingbuildOpenAISystemPrompt().yutoriBuiltinToolsOnPayloadalias;yutoriNativeToolSetOnPayloadis the single canonical name.toolDefinitions: []invariant in its module.provider-module.test.tsasserting every provider conforms to the contract; update the README to match.User-facing API changes (
@onkernel/cua-ai)<provider>.COMPUTER_TOOL_COORDINATES→<provider>.coordinateSystem()<provider>.build<Provider>SystemPrompt()for every provider (incl.buildOpenAISystemPrompt)yutori.yutoriBuiltinToolsOnPayload→ useyutori.yutoriNativeToolSetOnPayloadproviderModuleexport per provider and theCuaProviderModuletypeTests
npx tsc -b(whole monorepo): greennpm test --workspace @onkernel/cua-ai: 56 passing, 5 skipped (live-only)computer-tool.integration.test.ts): openai, anthropic, google, tzafon pass; yutori untested due to an ongoing upstreamn1 APIpartial outage (per yutori.statuspage.io), reproduced outside this code via raw HTTP — not a regression.